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Mr.  Chairman  and  Members  of  the  Subcommittee: 


Thank  you  for  the  opportunity  to  testify  on  the  National  Weather  Service’s 
(NWS)  Advanced  Weather  Interactive  Processing  System  (AWIPS) — the 
“linchpin”  of  NWS’  $4.5  billion  modernization  program.  The  weather 
service  modernization  entails  building  and  putting  into  place  vastly  more 
capable  weather  observing  systems,  such  as  Doppler  radars,  that  will  feed 
a  network  of  sophisticated  AWIPS  workstations.  This  AWIPS  network,  in 
turn,  is  to  use  the  observations  in  combination  with  national  weather 
modeling  results  to  aid  forecasters  in  making  and  communicating  localized 
weather  predictions.  NWS  estimates  that  AWIPS  will  cost  $525  million  to 
fully  deploy  by  1999. 

Mr.  Chairman,  today  we  are  the  bearers  of  both  good  and  bad  news,  some 
of  which  we  pointed  out  last  February  during  testimony  before  this 
Subcommittee.  The  good  news  is  that  NWS  has  done  a  stellar  job  of 
involving  its  forecaster  community  in  defining  what  AWIPS  should  be. 
Heeding  the  advice  of  a  1994  independent  review  team,  NWS  has  also 
reorganized  the  program  to  address  fundamental  management 
impediments  to  establishing  an  acceptable  system  design  and  moving  the 
program  forward.  Additionally,  NWS  has  acted  on  some  of  our  recent 
recommendations  to  strengthen  its  in-house  software  development 
capabUify. 

The  bad  news  is  that  NWS  runs  the  risk  of  wasting  money  on  AWIPS 
capabilities  that  may  not  be  needed  because  it  has  yet  to  link  aU  planned 
capabilities  to  promised  mission  improvements,  such  as  cheaper 
operations  and  better  forecasts.  Additionally,  NWS’  ability  to  meet  its 
AWIPS  commitments  is  being  jeopardized  by  a  risky  development 
approach  that  (1)  prematurely  begins  developing  one  software  increment 
before  the  previous  increment  is  stabilized,  (2)  complicates  government 
versus  contractor  accountability  for  inevitable  system  integration  and 
performance  problems,  and  (3)  omits  a  vital  development  process  known 
as  software  quality  assurance.  Also,  NWS  expectations  for  staffing 
reductions  from  the  modernization  continue  to  shrink. 


AWIPS:  A  Brief 
Description 


AWIPS  is  to  serve  as  both  a  weather  decision  support  system  and 
communication  system.  More  specifically,  AWIPS  is  to  support  forecasters 
in  graphically  integrating  and  analyzing  the  volumes  of  weather 
observations  and  products  that  form  the  basis  for  decisions  on  each  day’s 
forecasts  and  warnings.  It  is  also  to  provide  the  national  communications 
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infrastructure  for  NWS  field  offices  and  national  centers,  connecting  them 
not  only  to  each  other  but  also  linking  them  to  NWS’  diverse  customer 
base.  Through  AWIPS,  NWS  expects  to  tap  a  reservoir  of  data  from  its  new 
observing  systems  that  the  aging  processing  and  communication  system 
currently  in  place,  known  as  AFOS,  cannot.^ 

AWIPS’  progress  to  date  has  been  uneven.  Despite  early  successes  in 
effectively  involving  forecasters  in  AWIPS  requirements  analysis  and 
definition  activities  and  demonstrating  the  technical  feasibility  of  AWIPS 
functions,  AWIPS  made  Utfie  progress  during  1993  and  1994  because  of  an 
impasse  with  the  development  contractor  over  the  AWIPS  design  and 
shortcomings  in  NWS’  program  management.  Acting  on  the 
recommendations  of  an  independent  review  team,  NWS  was  able  to  move 
the  program  forward  in  1995  by  restructxuing  the  program  and 
renegotiating  the  development  phase  of  the  AWIPS  contract. 

Renegotiation  of  the  deployment  phase  of  the  contract  is  ongoing. 

Under  the  restructuring,  NWS  assumed  responsibility  for  developing  all 
AWIPS  hydrology  and  meteorology  application  software.  Also, 
development  of  the  system  was  divided  into  a  series  of  seven  increments 
of  increasing  functional  capability.  ’Thus  far,  the  first  increment  has  been 
installed  at  three  sites  to  gain  experience  in  developing,  testing,  deploying, 
and  operating  a  very  limited  version  of  AWIPS.  Development  of  the  second 
increment  is  underway. 

NWS’  current  project  cost  estimate  for  AWIPS,  which  was  first  reported  in 
December  1994  and  according  to  NWS  is  stiU  valid,  is  $525  million  or 
roughly  $58  million  more  than  its  previous  official  estimate  done  in 
October  1992.  The  current  schedule  calls  for  AWIPS  deployment  to  be 
completed  in  1999  or  1  year  later  than  NWS  projected  in  1992. 


How  Much  AWIPS 
Capability  Is  Enough? 


Since  its  inception  in  the  early  1980s,  NWS  has  justified  the  modenuzation, 
and  its  component  systems,  on  the  grounds  that  it  will  produce  significant 
“service-to-the-public”  improvements — namely,  better  forecasts  at  less 
cost.  To  facilitate  attaining  these  goals,  NWS  has  specified  that  AWIPS 
must  provide  about  450  high-order  capabilities,  such  as  the  ability  to 
execute  certain  models  or  display  data  in  certain  formats  and  colors.  AU 
told,  these  high-order  capabilities  are  composed  of  about  22,000  separate 
system  requirements. 


'AFOS  stands  for  Automation  of  Reid  Operations  and  Services. 
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AWIPS  Software 
Development  Risks 
Remain 


In  defining  AWIPS’  capabilities,  NWS  effectively  solicited  and  incorporated 
the  views  of  the  forecaster  community.  In  this  regard,  it  performed 
multiple  requirements  analyses  and  reviews  and  it  effectively  employed 
system  prototyping  to  ensure  that  forecaster  perspectives  were  heard  and 
understood. 

However,  the  true  measure  of  AWIPS’  value  is  not  that  forecasters  like  it, 
but  rather  that  it  contributes  significantly  to  improving  NWS’  mission 
performance.  In  studying  the  practices  of  leading  public  and  private  sector 
organizations,  we  found  that  successful  organizations’  system  investments 
are  based  on  exphcit  and  quantifiable  mission  improvements.  By  doing  so, 
these  organizations  know  that  investing  in  system  c^abihties  is  justified 
and  wiU  make  a  difference  in  mission  outcomes,  such  as  service  dehvery 
or  product  quality.^ 

Unfortunately,  NWS  has  not  demonstrated  that  the  package  of  capabilities 
it  envisions  for  AWIPS  will  enable  it  to  make  better  forecasts,  operate 
fewer  field  offices,  and  reduce  staffing  levels,  leaving  the  question  wide 
open  as  to  whether  AWIPS,  as  envisioned,  is  the  “right  thing.”  In  our  view, 
unless  NWS  takes  advantage  of  ongoing  and  planned  AWIPS  prototyping 
to  validate  that  proposed  c^abilities  produce  measurable  mission 
improvements,  it  runs  the  risk  of  wasting  taxpayer  money. 


Our  recent  work  on  the  NWS  modernization  has  identified  several  AWIPS 
development  risks,  particularly  with  regard  to  the  system’s  software. 
Despite  the  fact  that  NWS  has  moved  to  mitigate  some  of  these  risks, 
others  remain  that  require  careful  management  attention  and  action.  The 
risks  are  (1)  a  development  approach  that  is  predicated  on  overlapping 
software  builds,®  (2)  fines  of  accountability  between  the  government  and 
the  contractor  for  the  system’s  development  that  are  unclear,  and 
(3)  extensive  software  development  that  is  occurring  without  a  software 
quality  assurance  program.  Each  is  discussed  below. 

•  In  its  recent  restructuring  of  the  AWIPS  program,  NWS  responded  to  the 
recommendation  of  a  1994  independent  review  team  and  broke  the 
system’s  development  into  increments,  thus  employing  a  widely  accepted 
risk  reduction  strategy  of  “build-a-fittle,  test-a-fittle.”  Generally  speaking. 


^Executive  Guide;  Improving  Mission  Performance  Through  Strategic  Information  Management  and 
Technology  (GAO/AIMD-94-U5,  May  1994).  ~ 

'^Software  builds  are  a  series  of  software  increments,  each  with  increasing  capabilities,  that  add  to  or 
build  upon  the  capabilities  of  the  preceding  increment. 
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incremental  development  breaks  a  large,  monolithic  system  development 
effort  into  several  smaller,  more  manageable  development  pieces,  thereby 
permitting  earlier  warning  of  significant  system  development  problems, 
and  avoiding  expending  the  huge  sums  of  money  associated  with 
developing  a  complete  system  before  more  basic  capabilities  are 
successfully  developed. 

The  key  to  effective  incremental  development,  however,  is  to  ensure  that 
each  software  increment  or  build  is  stabilized  (i.e.,  free  of  material 
defects)  before  adding  new  capabilities  (i.e.,  software  components)  to  it 
during  succeeding  builds.  Without  build  stabilization,  new  software 
problems  are  introduced  on  top  of  already  existing,  unresolved  problems, 
greatly  increasing  the  time  and  money  needed  to  produce  mature 
software.  In  February  1995,  we  testified  that  NWS  was  not  providing  itself 
the  opportunity  for  AWIPS  build  stabilization  because  it  had  chosen  to 
overlap  its  software  builds  as  a  means  of  schedule  compression  to  meet 
arbitrary  deployment  dates.**  Specifically,  NWS  plans  to  enhance  and 
extend  AWIPS  software  components  (i.e.,  capabilities)  before  these  more 
basic  cj^abfiities,  upon  which  the  enhancements  and  extensions  will  rely, 
are  fuUy  developed  and  tested.  Without  a  mature  baseline  to  begin  each 
build,  existing  software  defects  are  likely  to  be  compounded,  causing  the 
time  and  money  needed  to  complete  AWIPS  to  grow. 

NWS  officials  agree  that  overlapping  AWIPS’  software  builds  is  a  risk. 
However,  they  stated  that  this  risk  wiU  be  mitigated  by  completely  testing 
one  build  before  movmg  on  to  the  next.  Our  analysis  of  the  AWIPS  build 
schedule  does  not  support  these  statements,  revealing  that  software  builds 
are  scheduled  to  begin  before  the  previous  build  has  been  stabihzed  (i.e., 
fully  tested  and  debugged). 

While  we  appreciate  and  share  NWS’  desire  to  field  AWIPS  capabilities  as 
soon  as  possible,  thereby  allowing  it  to  take  full  advantage  of  its  new 
observing  systems’  data  sets,  we  believe  that  overtyping  AWIPS’  software 
builds  introduces  an  element  of  risk  that  could  ultimately  slow  the 
system’s  completion  rather  than  accelerate  it,  not  to  mention  raise  its  price 
tag. 

•  In  January  1993,  we  reported  on  several  AWIPS  risks  confronting  NWS, 
including  unclear  roles  and  responsibilities  between  the  government  and 


^Weather  Service  Modernization:  Despite  Progress,  Significant  Problems  and  Risks  Remain 
(GAO/T-AIMD-95-87,  Feb.  21, 1995). 
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the  contractor.®  Again  last  year,  prior  to  AWIPS’  restructuring,  we  testified 
that  this  risk  remained.®  Under  AWIPS’  recent  restructuring  and  the 
associated  renegotiated  development  contract,  the  government  has 
maintained  a  large  software  development  responsibility,  writing  all  the 
hydrology  and  meteorology  applications,  while  the  contractor  delivers  the 
AWIPS’  hardware,  systems  software,  and  communications  networks  and 
integrates  these  with  the  ^phcations.  The  government’s  exposure  to  risk, 
however,  still  remains  because  it  is  uncertain  whether  the  government  or 
the  contractor  will  bear  responsibiUty  for  resolving  any  software  defects 
discovered  dining  system  integration  that  are  not  readily  attributable  to 
either  party. 

•  In  December  1994,  we  reported  that  NWS’  in-house  software  development 
processes  were  not  adequate  for  anything  more  than  the  AWIPS 
prototyping  activities  that  NWS  planned  for  itself  at  theit  time.^  As  just 
mentioned,  however,  NWS  has  since  assumed  responsibility  for 
developing  over  one-half  of  AWIPS’  1.5  mMon  lines  of  code,  thus  making 
its  need  for  mature  internal  software  development  process  cjqiabilities 
absolutely  vital. 

While  NWS  has  reported  taking  a  number  of  steps  to  strengthen  its 
software  development  processes,  such  as  establishing  a  software 
development  plan,  we  are  aware  of  at  least  one  serious  process  weakness 
that  remains.  Namely,  NWS  has  not  established  a  software  quality 
assurance  program  for  AWIPS.  In  a  nutshell,  software  qualify  assurance 
exists  to  address  the  management  axiom  of  “what  is  not  tracked  is  not 
done.”  Such  a  program  independently  (1)  monitors  whether  the  software 
and  the  processes  used  to  develop  it  fully  satisfy  estabUshed  standards  and 
procedures  and  (2)  ensures  that  any  deficiencies  in  the  software  product, 
process,  or  their  associated  standards  are  swiftly  brought  to  management’s 
attention.  In  our  view,  the  absence  of  a  software  quaUfy  assurance 
program  for  AWIPS  exposes  the  project  to  unacceptable  cost,  schedule, 
and  performance  risk. 


^Weather  Forecasting:  Important  Issues  on  Automated  Weather  Processing  System  Need  Resolution 
(GAO/IMTEC-93-12BR,  Jan.  6,  1993). 

^Weather  Service  Modernization:  Despite  Progress,  Significant  Problems  and  Risks  Remain 
(GAO/T-AIMD-95-87,  Feb.  21,  1995). 

^Weather  Forecasting:  Improvements  Needed  in  Laboratory  Software  Development  Processes 
(GAO/AIMD-95-24,  Dec.  14,  1994). 
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Originally  Promised 
Staffing  Reductions 
Will  Not  Materialize 


Among  the  mission  benefits  to  be  derived  firom  the  NWS  modernization 
was  a  21-percent  reduction  in  its  then  current  staffing  levels — a.  drop  from 
5,100  to  4,028.  However,  in  September  1995,  we  reported  that  the  4,028 
target  staffing  level  had  grown  to  4,678— a  decrease  in  projected  staff 
savings  of  650  or  61  percent.®  NWS  attributed  the  reduction  in  expected 
staff  savings  to  underestimating  the  number  of  staff  needed  to  operate  and 
maintain  the  new  systems  as  well  as  unexpected,  external  direction  to  add 
field  offices  and  perform  specialized  forecasting  activities.  As  a  result  of 
the  Secretary  of  Commerce’s  October  1995  decision  for  NWS  to  add  one 
more  field  office  and  three  NEXRAD  sites,  expected  staffing  savings  will 
decrease  even  more,  perhaps  by  as  much  as  60,  according  to  NWS 
officials.  Such  a  staffing  change  would  increase  NWS’  target  staffing  level 
to  4,738,  which  is  710  more  than  the  original  target  levels. 


In  conclusion,  the  inadequacies  of  AFOS  and  the  potential  utility  of 
incorporating  new  observing  systems’  data  sets  into  forecast  models  and 
analyses  argue  strongly  for  an  AWIPS-like  system  to  support  NWS 
decision-making  and  communications  needs.  However,  because  NWS  has 
not  hnked  AWIPS  capabilities  to  exphcit,  measurable  improvements  in 
mission  performance,  we  do  not  know  whether  AWIPS  as  currently 
defined  with  all  its  capabUities  is  a  wise  investment.  Furthermore,  because 
of  continuing  software  development  risks,  it  is  uncertain  that  NWS  will 
deliver  AWIPS  as  promised. 

Mr.  Chairman,  this  concludes  my  statement.  We  will  be  happy  to  respond 
to  any  questions  you  or  Members  of  the  Subcommittee  might  have  at  this 
time. 


^Weather  Service  Modernization  Staffing  (GAO/AIMD-95-239R,  Sept.  26,  1995). 
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